[Shopify] Migrate Shopify Connector to Expiring Offline Access Tokens by onbuyuka · Pull Request #9327 · microsoft/BCApps

onbuyuka · 2026-07-11T14:44:52Z

Background

As of December 2025, Shopify supports expiring offline access tokens with refresh-token rotation. Public apps created before 2026-04-01 must migrate to expiring tokens by 2027-01-01; after that date, REST/GraphQL Admin API requests made with non-expiring tokens are rejected. The Shopify Connector currently uses non-expiring offline tokens and must be migrated.

Token characteristics (Shopify):

Access token lifetime: 1 hour (expires_in: 3600)
Refresh token lifetime: 90 days (refresh_token_expires_in: 7776000)
Migration is a one-time, irreversible token exchange per shop (the original non-expiring token is revoked on success)
Only one refreshable expiring token per app + store

Refs: About offline access tokens - Migrating from non-expiring to expiring tokens

Approach

Token validity is centralized in a single orchestrator, EnsureValidAccessToken(Store), invoked from Shpfy Communication Mgt.GetAccessToken - so every API call (interactive and background) transparently migrates a legacy token or refreshes an expiring one before use. A scheduled backstop job additionally keeps idle shops access/refresh tokens alive and completes migrations for shops that never make an on-demand call.

Changes

Token storage - `Shpfy Registered Store New` (table 30138)

New fields Token Expires At / Refresh Token Expires At; refresh token kept in IsolatedStorage.

Acquisition, refresh & migration - `Shpfy Authentication Mgt.` (codeunit 30199)

New installs request expiring tokens (expiring=1).
EnsureValidAccessToken orchestrates migrate-if-legacy / refresh-if-near-expiry, serialized per shop.
RefreshAccessToken (grant_type=refresh_token) with transient retry and a terminal reconnect error.
MigrateToExpiringToken (one-time token exchange, best-effort - keeps the working token on failure).
ForceTokenRefresh, IsRefreshTokenExpired, and a Shop Card reconnect notification.

Consumption - `Shpfy Communication Mgt.`

GetAccessToken invokes the orchestrator; reactive 401 -> refresh -> retry in ExecuteWebRequest.

Scheduled backstop - `Shpfy Token Refresh` (30431) + `Shpfy Token Refresh Shop` (30432)

Dispatcher (TableNo = Job Queue Entry) iterates enabled shops and runs the per-shop worker (TableNo = Shpfy Shop) via Codeunit.Run for isolation; recurring Job Queue entry registered via installer + upgrade.

UX - `Shpfy Shop Card`

Reconnect notification when the refresh token has expired.

Docs & tests

Shpfy Token Refresh Test unit tests; living docs refreshed (business-logic, data-model, patterns, CLAUDE.md).

Telemetry

Emitted for migration success/failure, refresh success/transient/terminal and backstop per-shop failures, under event IDs 0000UIV-0000UJ1.

Test plan

Shopify App project compiles clean.
Shpfy Token Refresh Test unit tests for the expiry decisions.
Install / refresh / migration / backstop / notification scenarios validated locally.

Fixes AB#637954

Adds support for Shopify expiring offline access tokens (public apps must migrate by 2027-01-01): - Persist token/refresh expiry on "Shpfy Registered Store New" and store the refresh token in IsolatedStorage. - Request expiring tokens on install (expiring=1), refresh before expiry, and migrate legacy non-expiring tokens via token exchange (best-effort). - Orchestrate on-demand from "Shpfy Communication Mgt." (GetAccessToken) with a reactive 401 refresh-and-retry. - Scheduled backstop job ("Shpfy Token Refresh" + per-shop worker) to keep tokens and 90-day refresh tokens alive; registered via installer/upgrade. - Shop Card reconnect notification when the refresh token has expired. - Tests for the refresh-token expiry decision logic. NOTE: "Shpfy Token Dev Tools" (page 30440) is temporary test scaffolding and must be removed before merge. Fixes AB#637954 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

- Remove the temporary "Shpfy Token Dev Tools" page (30440) and its permission-set entry. - Clear the new telemetry event IDs (set to '') so they can be assigned by the tagging script. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

Refresh the living docs (al-docs update) to cover slice 637954: - business-logic.md: new "Authentication and token lifecycle" section. - data-model.md: Shpfy Registered Store New token/expiry storage + ER entry. - patterns.md: Job Queue dispatcher/worker per-shop isolation pattern. - CLAUDE.md (app + Base): auth overview and "things to know". Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

Remove the LogTokenTelemetry helper and inline Session.LogMessage('', ...) at each call site so the telemetry tagging script can assign event IDs (it only fills literal Session.LogMessage first-args). Matches the existing connector convention. Tags left empty pending the script. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

Fill the token lifecycle telemetry event IDs (0000UIV-0000UJ1) via the tagging script. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

- Mark SaveInstalledToken [NonDebuggable] (it holds the token-bearing response body). - Give the token-refresh backstop its own Job Queue category (SHPFYAUTH) so it is not serialized behind long-running SHPFY sync jobs (pricing/inventory) and can refresh 1-hour access tokens promptly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

github-actions · 2026-07-13T07:08:49Z

Copilot PR Review

Iteration 11 · Outcome: completed

Knowledge source: https://github.com/microsoft/BCQuality@186d8a131465475c79244d994acb872cd5c0d4bf

Findings by domain

Findings split into Knowledge-backed (cite a BCQuality article) and Agent (the agent's own judgement, no matching BCQuality rule).

Domain	Findings	Knowledge-backed	Agent	Inline	Fallback
Error Handling	1	1	0	1	0
Performance	1	1	0	0	0

Totals: 2 knowledge-backed · 0 agent findings.

Orchestrator pre-filter (2 file(s) excluded)

layer-disabled (knowledge) : 2 file(s)

_{Findings produced by the AL review agent v1.7.3. Reply 👎 on any inline comment to flag false positives.}

- Encrypt the refresh token at rest: IsolatedStorage.SetEncrypted instead of Set (90-day credential; Get transparently decrypts). - Mark ResponseHasAccessToken [NonDebuggable] (handles the token-bearing body). - Remove the redundant per-iteration Commit() in the token-refresh dispatcher loop; the per-shop worker (Codeunit.Run) already commits on success. - Guard Codeunit.Run("Job Queue - Enqueue") in ScheduleRefreshJob so an enqueue failure is logged instead of aborting the install/upgrade. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

github-actions · 2026-07-13T08:39:13Z

$\textbf{🟠\ High\ Severity\ —\ Security} \quad \color{gray}{\texttt{\small Iteration\ 2}}$

SaveTokenResponse (new in this PR) stores the Shopify offline access token via RegisteredStoreNew.SetAccessToken(), which persists it with IsolatedStorage.Set (unencrypted at rest), while the refresh token introduced by this same PR is correctly stored via SetRefreshToken() using IsolatedStorage.SetEncrypted.

Both are OAuth credentials for the same store and belong in the same encrypted class. This inconsistency becomes load-bearing now: every store going forward will have an encrypted refresh token sitting next to an unencrypted, equally sensitive access token in the same record. Change SetAccessToken (Shpfy Registered Store New table) to use SetEncrypted for parity with the refresh token.

Knowledge:

microsoft/knowledge/security/isolatedstorage-setencrypted-for-sensitive-values.md

Line mapping was unavailable, so this was posted as an issue comment.

_{👍 useful · ❤️ especially valuable · 👎 wrong - reply with why}

Fill 0000UJ6 (enqueue-failure warning) via the tagging script. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

A legacy store has no refresh token, so EnsureValidAccessToken never hits the fast path and re-attempts migration on every API call -- in a sync loop over many records a failing migration means many redundant lock+HTTP+commit round trips. Add a "Last Migration Attempt" timestamp on Shpfy Registered Store New and only re-attempt migration once per hour (ShouldAttemptMigration/TryMigrate, shared by EnsureValidAccessToken and ForceTokenRefresh). Docs updated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

- Do not interpolate GetLastErrorText() into the telemetry message string (LogRefreshFailure and the schedule-failure log). Use a generic message, move the error detail and shop code to custom dimensions, and classify the event as CustomerContent so the platform can handle it appropriately. - Move the shared telemetry Labels to the codeunit object scope. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

Codeunit 139635 is already used by "E-Doc. Receive Files" in the EDocument test app; BCApps validates test object IDs globally. Move Shpfy Token Refresh Test to 139613, which is free repo-wide within the Shopify test ID ranges. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

…riant API Test) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

CreateReconnectErrorInfo embeds the store domain (customer-bearing) in the message and a custom dimension, and ErrorInfo content flows to telemetry, so DataClassification must be CustomerContent - consistent with LogRefreshFailure in the same codeunit - rather than SystemMetadata. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

…ation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

github-actions · 2026-07-20T11:37:48Z

$\textbf{🟠\ High\ Severity\ —\ Privacy}$

The new token refresh flow posts to Shopify without checking a custom Privacy Notice approval state, and the Shopify app has no privacy-notice registration anywhere under the app. That leaves this new background external-transfer path issuing shop-specific auth requests without the integration-specific consent required by the privacy guidance.

Knowledge:

Line mapping was unavailable, so this was posted as an issue comment.

_{👍 useful · ❤️ especially valuable · 👎 wrong - reply with why · AL review agent v1.12.4}

github-actions · 2026-07-20T11:37:49Z

$\textbf{🟠\ High\ Severity\ —\ Security}$

ExecuteTokenRequest builds the OAuth token-request URL with StrSubstNo(AccessTokenURLTxt, Store), then posts the client secret and refresh/access-token material to it, without ever calling IsValidHostName/AssertValidShopUrl (both already defined in this codeunit) on Store first. This PR newly routes two additional call sites through this helper with a table-sourced Store value: the scheduled backstop refresh (via ShpfyTokenRefreshShop -> Rec.GetStoreName()) and the reactive 401 refresh (via ShpfyCommunicationMgt.HandleUnauthorizedResponse -> Shop.GetStoreName()). If a Shpfy Shop row's stored URL is ever malformed or tampered with, credentials are posted to whatever host that field contains with no revalidation at the point of use.

Suggested fix (apply manually — could not be anchored as a one-click suggestion):

        if not IsValidHostName(Store) then
            Error(InvalidShopUrlErr);
        Url := StrSubstNo(AccessTokenURLTxt, Store);

Knowledge:

microsoft/knowledge/security/validate-user-configurable-urls.md

Line mapping was unavailable, so this was posted as an issue comment.

_{👍 useful · ❤️ especially valuable · 👎 wrong - reply with why · AL review agent v1.12.4}

Token-refresh/migration failure branches (migration failed, refresh-token expired, transient-exhausted) were logged at Verbosity::Normal, which hides them from severity-based alerting; raise them to Warning (success events stay Normal). Also SetLoadFields for the Shopify URL in RefreshAllShops since the per-shop worker only needs the store URL, avoiding full-record transfer of the wide Shop table on the recurring job. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

…e tag Three safe PR-review fixes: - 401 circuit breaker: ForceTokenRefresh now records a per-store timestamp and skips a repeat forced refresh within a one-minute cooldown, so a store that returns 401 for a non-expiry reason no longer re-refreshes (and re-serializes on the row lock) once per record during a bulk sync. It returns whether a refresh was attempted so the caller only retries when it was. - Transient-failure error: when refresh fails only on transient 5xx/network and the access token is already expired, raise a non-actionable "try again later" error instead of the reconnect fix-it action, which is not the real remedy. - Upgrade tag: ScheduleRefreshJob returns success; the upgrade step only records its tag when enqueuing succeeded, so a transient enqueue failure retries on a later upgrade run instead of permanently suppressing the backstop. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

…637954-migrate-shopify-expiring-tokens

github-actions · 2026-07-20T12:11:40Z

$\textbf{🟡\ Medium\ Severity\ —\ Upgrade}$

OnInstallAppPerCompany() schedules the token-refresh job, but the install path never marks MS-637954-ScheduleTokenRefreshJob-20260711 as completed. That leaves companies installed directly on this version without the historical upgrade tag, so the next app upgrade will re-enter ScheduleTokenRefreshJobUpgrade() instead of recognizing that install already performed the step. Per the referenced guidance, install-time initialization that replaces a later upgrade step should explicitly set the same upgrade tag.

Knowledge:

microsoft/knowledge/upgrade/register-upgrade-tags-with-subscribers.md

Line mapping was unavailable, so this was posted as an issue comment.

_{👍 useful · ❤️ especially valuable · 👎 wrong - reply with why · AL review agent v1.12.4}

…l loudly on token exchange - Circuit-breaker cooldown: ForceTokenRefresh committed the Last Force Refresh At marker only after RefreshAccessToken returned, so a terminal reconnect error rolled it back and every subsequent 401 re-entered the refresh. Commit the cooldown before the refresh attempt so the throttle persists on the failing path. - Reconnect ErrorInfo: set Title and DetailedMessage alongside the existing Reconnect action so the actionable error has a clear heading and supporting detail. - Initial OAuth token exchange: GetToken exited silently on a non-success status or a payload without an access token, so Request Access/reconnect failed with no feedback. Raise a specific error instead. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

…all/upgrade Enqueuing a Job Queue Entry runs Codeunit.Run("Job Queue - Enqueue"), which implicitly commits on success (and ScheduleTask commits to register the task). Implicit commits are not allowed inside install/upgrade triggers, so scheduling the backstop from OnInstallAppPerCompany / OnUpgradePerCompany crashed a fresh install with "the transaction is stopped". Move scheduling out of the install/upgrade transaction entirely: schedule the recurring job lazily from ShpfyCommunicationMgt.GetAccessToken (the single chokepoint for all connector API usage), once per session via a SingleInstance guard. This runs in a normal committable runtime context and uniformly covers install, upgrade, new companies, and pre-existing shops. Removed the install and upgrade scheduling calls and the now-unused upgrade tag. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

github-actions Bot added the AL: Apps (W1) Add-on apps for W1 label Jul 11, 2026

github-actions Bot added this to the Version 29.0 milestone Jul 11, 2026

onbuyuka and others added 4 commits July 11, 2026 17:06

[Shopify] Assign telemetry event IDs

c0e21fd

Fill the token lifecycle telemetry event IDs (0000UIV-0000UJ1) via the tagging script. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc

AndreiPanko reviewed Jul 11, 2026

View reviewed changes

Comment thread src/Apps/W1/Shopify/App/src/Integration/Codeunits/ShpfyAuthenticationMgt.Codeunit.al

Comment thread src/Apps/W1/Shopify/App/src/Integration/Codeunits/ShpfyTokenRefresh.Codeunit.al

onbuyuka marked this pull request as ready for review July 13, 2026 06:57

onbuyuka requested a review from a team July 13, 2026 06:58

github-actions Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread src/Apps/W1/Shopify/App/src/Integration/Codeunits/ShpfyAuthenticationMgt.Codeunit.al

github-actions Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread src/Apps/W1/Shopify/App/src/Integration/Codeunits/ShpfyTokenRefresh.Codeunit.al Outdated

github-actions Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread src/Apps/W1/Shopify/App/src/Integration/Tables/ShpfyRegisteredStoreNew.Table.al Outdated

github-actions Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread src/Apps/W1/Shopify/App/src/Integration/Codeunits/ShpfyTokenRefresh.Codeunit.al Outdated

darjoo requested changes Jul 13, 2026

View reviewed changes

Comment thread src/Apps/W1/Shopify/App/src/Integration/Codeunits/ShpfyAuthenticationMgt.Codeunit.al

github-actions Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread src/Apps/W1/Shopify/App/src/Base/Codeunits/ShpfyCommunicationMgt.Codeunit.al Outdated

github-actions Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread src/Apps/W1/Shopify/App/src/Integration/Codeunits/ShpfyAuthenticationMgt.Codeunit.al

onbuyuka and others added 2 commits July 13, 2026 10:50

[Shopify] Assign telemetry event ID for token-refresh schedule failure

5e8ec0d

Fill 0000UJ6 (enqueue-failure warning) via the tagging script. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc